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ABSTRACT 

This prevention training program was created in 
response to the emergency situation in Community School District 3 in 
New York City. The program goal was to provide intensive staff 
development to 275 school personnel, from all segments of the 
educational community, in the skills and information necessary to 
provide substance abuse prevention education to students and to 
mobilize parents in preventive efforts. Training was to be evaluated 
through administering pre- and post-tests to staff participants. 
In-class observations were also to be conducted to assess the extent 
to which staff members were able to translate program training into 
lessons for students. The primary means of achieving proposal goals 
were through workshops and training programs. The content of staff 
development activities included information on substance abuse and 
related issues, strategies for building self esteem, student 
activities designed to encourage critical thinking about drugs and 
sociopolitical issues, and ways of enabling students to pursue 
creative activities. Much of the material was age-specific and 
culturally sensitive. Data submitted to the Office of Educational 
Research (OER) , for program evaluation were survey material from 
workshops and records of program activities. (BF) 
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EXECUTIVE SUMMARY 

PROJECT BACKGROUND 



T,ie Expanded and Enhanced Model System Wide 7-9 Drug and 
Alcohol Abuse Prevention Training Program (1991-1992) was a 
federally funded project created under the Drug-Free Schools and 
Communities Program Emergency Grants in response to the emergency 
situation in Community School District #3. The proposal goal was 
to provide intensive staff development to 275 school personnel in 
the skills and information necessary to provide substance abuse 
prevention education to students and to mobilize parents in 
preventive efforts. The district's philosophy is that a holistic 
perspective must be taken integrating family and developmental 
issues in substance abuse in preventive education. All segments 
of the educational community , i.e.: instructional, non- 
instructional , and support staff were to be involved in the 
preventive effort. 

Program sponsored training was to be evaluated through 
administering pre-and post- tests to staff participants • In 
addition , in-class observations were to be conducted to assess 
the extent to which staff members were able to translate program 
training into lessons for students. 

PROJECT FINDINGS 

The primary means of achieving proposal goals were through 
workshops, and training programs. Eighty eight staff development 
activities were held in which 1120 staff members participated*. 
The actual number of participants who received training greatly 
exceeded the proposed numbers. 

The content of staff development activities met the program 
objectives. They included information on substance abuse and 
related social „ health, and developmental issues, strategies for 
providing emotional support and building self-esteem, student 
activities designed to encourage critical thinking about drugs 
and sociopolitical issues, and ways of enabling students to 
pursue creative activities. Much of the material was age- 
specific , and culturally sensitive, as specified in the proposal 
guidelines. 

Feedback about the workshops was furnished through 
evaluation forms. In general, workshop activities were highly 
rated. Staff described the workshops attended to be useful in 
the knowledge and group support provided, and most stated that 
they came away better aware of the problems related to substance 
abuse, and of what they could do to address them. Over 50 
percent of the participants gave the workshops excellent ratings 
in all areas. Staff participants at workshops requested that 
workshops be more structured, contain material relevant to their 
student populations, and include discussion of strategies by 



* Participants in more than one activity were counted more 
than once. 



which information presented can be practically applied. 

Data submitted to the Office of Educational Research 
(O.E.R.), program evaluation were survey material from 

workshops and records of program activities. After reviewing 
these materials O.E.R. found that the assessment measures 
outlined in the proposal were not utilized. Neither were 
criterion referenced pre- and post- tests nor were program 
developed observation checklists used. Therefore, survey 
responses and records of program activities form the basis of the 
O.E.R. evaluation. 



CONCLUSIONS AND RECOMMENDATIONS : 

The program objectives were largely met in terms of the 
concepts addressed and the numbers and kinds of activities held. 
O.E.R. recommended the following changes to enhance the second 
year of program implementation: 

(1) As specified in the proposal, pre and post criterion 
referenced tests should be administered to evaluate the 
knowledge that participants obtained through program 
participation. 

(2) As specified in the proposal, staff trainees should be 
observed in order to assess the success of training in 
applied situations. 

(3) Staff development training attendance sheets should 
have a space designated for participants' job titles, 
in order to determine categories of staff trained. 

(4) District staff should provide technical assistance to 
school staff so that the information from workshops 
can be readily applied in concrete situations. 

(5) Information should be provided on the extent to which 
the services of community providers are being utilized 
by staff and students. 
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I . INTRODUCTION 
This report documents the Office of Educational Research's 
(O.E.R.'s) evaluation of the first year (1991-92) of the two-year 
drug abuse prevention and education grant for the Expanded and 
Enhanced Model System-Wide 7-9 Drug/Alcohol Abuse Prevention 
Training Program awarded to Community School District 3 (CSD 3) . 
The main objective of the 7-9 Substance Abuse Prevention Program, 
1991-93, was to reduce substance abuse through staff development. 
Two hundred and seventy-five school staff were targeted for the 
first year of program implementation. Unlike previous substance 
abuse prevention and education staff development programs 
developed by CSD 3, this program targeted instructional, non- 
instructional staff, and support staff such as custodial workers, 
lunchroom workers , school aids, security staff, and school 
secretaries *. 
OVERVIEW OF DISTRICT 

This program was developed in response to the acute needs of 
the community school district feeder neighborhoods. Both the 
percentage of youth arrested and convicted of sufostance-abuse- 
related crimes and the number of youths referred for alcohol and 
drug abuse treatment and rehabilitation were strikingly high. 
According to the proposal, the total number of arrests of youths 
under 21 for substance-abuse-related crimes in the two police 



Individuals targeted for training include administrators, 
teachers, guidance counsellors, social workers, medical staff, 
school based substance abuse prevention and intervention 
specialists , librarians, and paraprof essionals . Additionally, 
the following five new groups of school employees will be 
targeted for staff development: school secretaries, school 
guards, school aides, custodial staff, and school lunchroom 
workers . 



precincts geographically encompassing the CSD was 1,622 for 
September, 199 0. This was nearly twice the city's average number 
of arrests per month for youths under 21 (849 per month) . 

As district provided data indicated, high percentages of the 
district's youth displayed the following risk characteristics: 
academic failure, dropping out of high school \ early sexuax 
activity with increased probabilities of teenage pregnancy and 
AIDS, mental/ emotional disorders, violence and crime. The 
majority of the students are economically disadvantaged with 
51.94 percent living below the federally defined poverty line and 
almost one third of the students are part of families eligible 
for Aid for Families with Dependent Children (A.F.D.C.). 
PROGRAM OBJECTIVES INDICATED IN THE PROPOSAL 

The overall program goals were to train staff to identify 
the causes of substance abuse and to provide instruction and 
guidance that enables students to resist and cope with risk 

factors. Expected staff outcomes are increased knowledge of and 

skills in providing substance abuse prevention education, and 

increased ability to provide students with opportunities to 

enhance their self-esteem. 

Staff De velopment Objectives 

According to the proposal, through program participation 

staff were to demonstrate increased knowledge and skills 

associated with: 

• an in-depth understanding of the substance abuse field 
including the pharmacological and psychosocial causes of 
drug and alcohol abuse; 
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* This district has one of the highest drop out rates in the 
nation. The official rate, according to NYC Public Schools data 
was 30.7 percent in 1990. 
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• culturally sensitive, accurate and age-appropriate 
knowledge about alcohol and other substances; 

• infusion of substance abuse prevention education into all 
curriculum areas; 

• the role of tobacco, marijuana, and alcohol as gateway 
drugs ; 

• awareness of how social attitudes and values contribute 
to the alcohol and drug abuse syndrome; 

• how to counter the "responsible use" message often 
promoted in the media; 

• building student's self-esteem; 

• promoting student success and developing appropriate life 
skills to resist drugs and alcdhol; 

• serving as effective role models; 

• understanding and addressing the needs of children of 
alcoholics; 

• helping students to say "No" without losing their 
friends; 

• training staff to serve as point-persons in each school; 

• developing meaningful opportunities for parent 
participation; 

• mobilizing community resources in support of substance 
abuse prevention education; 

• enabling students to identify and reduce personal risk 
factors and to manage stress without the use of 
drugs ; and 

• providing instruction in decision-making, and problem- 
solving skills. 



To achieve these objectives a number of staff development 
activities were to be implemented. These included: a ten-day 
summer training program, enrollment in specially designed college 
courses, participation in a series of workshops, as well as 
attendance at professional conferences conducted by the New York 
State Education Department and the Northeast Regional Center for 

3 
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Drug Abuse Prevention . Community-based substance abuse 
prevention education service providers were also to provide staff 
development* 

DISTRICT EVALUATION PLAN 
Methodology of Evaluation 

According to the proposal the purposes of the 1991-92 
project evaluation were to assess whether project implementation 
conforms to project design, and whether activities, materials, 
and instruments meet participant needs and project objectives* 
Evaluating Staff Development 

The following evaluation measures were to be used to 
determine whether staff development objectives were met: 

(a) Criterion referenced pre- and post-tests were to be 
administered and results compared. Success will be measured 
by a statistically significant increase in knowledge of drug 
and alcohol abuse prevention concepts, methods, and 
instructional/support service delivery strategies. 

(b) Classroom. observations were to be conducted by a 
qualified observer to test the teacher's ability to provide 
drug abuse prevention education* 

Evaluating Students 

According to the proposal, students will be evaluated only 
in the second year of program implementation. During the first 
year of the project, a cluster of student characteristics and 
outcome indicators were to be developed to assess the impact of 
staff development on students during the 1992-1993 school year. 
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II. O.E.R. EVALUATION 
O.E.R. EVALUATION MATERIALS AND METHODOLOGY 

The Office of Educational Research (O.E.R.) of the New York 
City Public Schools conducted an evaluation of the Expanded and 
Enhanced Model System 7-9 program activities over the period July 
1991-August 1992 to assess whether CSD 3 attained its objectives. 
In identifying the strengths and weaknesses in data collection 
O.E.R. hopes to enable the district to work toward improved 
evaluation techniques during the 1992-93 school year. 
Evaluation Material 

After a review of the data, O.E.R. found that the assessment 
measures outlined in the proposal were not utilized. Neither 
were criterion referenced pre and post tests used nor were 
classroom observation checklists administered. The only data 
submitted were survey materials from workshops and records of 
program activities. Therefore, these data formed the bases for 
O.E.R. 's evaluation. 
O.E.R. EVALUATION FINDINGS 
Overview 

Appendix tables 1 and 2 compare the planned project services 
to those implemented. Appendix table 2 delineates the target 
groups as well as the proposed and the actual number of persons 
served through program activities during the 1991-92 school year. 
Appendix table 2 indicates that 1,120 staff persons attended 
training which exceeds the estimated 275 from the proposal*. A 
high proportion of the proposed numbers of guidance counsellors, 

Staff who attended more than one workshop were counted 
more than once. 
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social workers, paraprof essionals , and teachers were trained • 
However, since staff did not give their title specifications on 
most workshop attendance forms, O.E.R. could not accurately 
determine the numbers of tftaff trained in each category- O.E.R. 
could not ascertain whether the five additional categories of 
staff — custodial workers, lunchroom workers, school aides, 
security staff, and school secretaries — received training. 

Appendix table 3 outlines program activities. In total 88 
training sessions were actually implemented. A variety of 
resources — including community based organizations, hospitals, 
and universities were sought out to develop a thorough 
training program encompassing all aspects of substance abuse 
prevention education. As proposed, the City University of New 
York conducted a number of training activities. In addition to 
collaborating with other organizations to provide services, 
project personnel themselves provided a variety of workshops on 
conflict-resolution, consciousness building, and group dynamics 
as proposed. The only kinds of projected staff activities that 
did not take place were presentations in conjunction with the New 
York State Education Department, and with the U.S. Northeast 
Regional Center for Substance Abuse Prevention. Additionally, we 
do not have information on whether staff visited community-based 
substance abuse prevention service providers as proposed. 

Staff development workshops addressed many of the program 
objectives. Workshops provided staff with knowledge of the 
psychological, physiological, and sociological aspects of 
substance abuse. In addition to workshops on substance abuse, 
workshops were held on child development, and on specific 
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psychological issues such as Child Abuse and Adolescent Attention 
Deficit Hyperactivity Disorder , and Post-Traumatic Strens 
Syndrome. Staff were trained to identify signs of addiction, and 
family based problems such as child abuse and substance abusing 
family members. Workshops provided referral information, and to 
the extent to which possible, staff were trained to address 
addiction and intervene with substance abusing families. 
Additionally, strategies for imparting refusal, coping, stress 
management, and communication skills to parents and students were 
discussed. 

An after-school professional development series* including 
classes and workshops were designed to assist staff in 
stimulating student interest in school. Staff development was 
provided in Communication Arts, Computer Education, Math, Science 
and Social Science Education, as well as in other subjects. The 
variety of classes offered was in line with the objective of 
servicing all kinds of staff. Up-to-date training was given to 
pupil personnel staff, and workshops on early childhood education 
were provided for elementary school teachers. 

Participant assessments of each of the staff development 
workshops for which evaluation forms were provided are detailed 
in appendix tables 4-10**. Sixty-four percent of participants in 
a random sample (N=50) described individual workshops to be above 
average or excellent in their effectiveness, and seventy percent 

Additional funding for these workshops was provided by the 
Drug-Free Schools and Communities Act Projects, Title II, PCEN 
and Chapter 1. 

Evaluation forms did not accompany all of the staff 
workshops . 
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gave workshops the same rating for their instructional value. In 
another random sample (N-68) 60 percent of the participants rated 
workshops excellent and 42 percent found the information 
presented to be new. Appendix tables 5 and 6 summarize these and 
other close-ended responses. 

Feedback from staff participants indicate that the workshops 
were well appreciated. Participants especially valued the 
information presented by the facilitator, the benefits of small 
group discussion where participants raised specific cases, the 
provision of resource and referral information, and the chance to 
meet key personnel. Responses to the question of what 
participants considered the most helpful aspect of the workshops 
included: "Getting to know each other", "A greater awareness of 
how to be with children", and "Being able to express myself", 
some of the comments on how participants thought the workshop had 
changed them included: "I will be more observant", "I will try to 
be a better listener" and "I will try harder to reach people and 

to help others". 

The presentations on AIDS, stress management, and the 
discussions of individual cases were greatly appreciated. The 
following were suggested future workshop topics: AIDS, family 
relationships, self-esteem building activities, communication 
techniques, substance abusing parents, and conflict resolution. 
Participants made the following recommendations to improve 
workshops : 

1 Workshops should be more structured and the speakers 
should increase their facilitation of group discussion. 

2. There should be more discussion of ways workshop 
information can be applied in concrete situations. 
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Guidance counsellors asked for more in-school non- 
clinical ways in which the material presented can be made 
relevant to counselling. 

3 . More training should be provided on how to communicate 
with students and parents, and on how to encourage them 
to express themselves freely. 

4. Workshop sessions should be longer with more follow-up 
workshops and follow through of specific cases that were 
discussed. 

Since no class-room observation data were submitted, O.E.R. could 
not determine whether trainees' skills in providing substance 
abuse prevention education increased. 
Pupil Outcomes 

According to proposal objectives outlined in Table 1, 3,7 50 
students were to be recipients of the Model System 7-9 program 
- activities. Program staff were to develop a listing of student 
characteristics to de* ermine the impact of staff development on 
students at the end of the 1992-93 school year. O.E.R. did not 
receive any data pertaining to the development of these pupil 
outcome indicators and therefore cannot assess them. 

Although the program did not call for pupil services, a 
number of activities were developed for students. These included 
workshops , extensive in-class substance abuse prevention training 
by SAPIS, a youth to youth mentoring program, and the extended 
day program which offered enriched instruction in a variety of 
subjects throughout the school year. Through the Youth to Youth 
Tutoring and Mentoring Program students in four elementary 
schools received tutoring from junior high school students during 
the school day and/or after-school . Eighty-four middle school 
tutors worked between February and May 1992 for a total of 2,182 
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hours*. Junior High School tutors received pre-service 
orientation training, on-going weekly training, and attended 
support group sessions. 

Workshops, extended day activities, and mentoring activities 
were in keeping with pupil services objectives. They provided 
students with a knowledge of substance abu e and related issues, 
and promoted psychological health, intellectual stimulation, and 
the building of community consciousness in students. 
O.E.R. Assessment of Program Material 

An O.E.R. researcher reviewed the project curriculum and 
other materials used in the project to determine .whether they 
were age-appropriate and culturally-sensitive as specified in the 
proposal. Staff development literature fully discussed how 
children of different age groups would respond to the various 
strategies and issues that were presented. Much of the project's 
literature reviewed by O.E.R. considered cultural differences and 
several staff workshops provided information about the history 
and sociology of non-European peoples. 



# The number of elementary school tutees was unspecified, 
O 10 
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II. CONCLUSIONS AND RECOMMENDATIONS 

CONCLUSIONS 

Overall, program activities provided a comprehensive 
substance abuse prevention education by addressing substance 
abuse in the context of its related sociological, psychological 
and physiological aspects, and by trying to involve all segments 
of the education community in a preventive effort. The program 
met its primary task of providing staff with knowledge of, and 
skills in substance abuse prevention education. The actual 
number of staff participants exceeded the numbers proposed. 
Program activities met proposal specifications in terms of 
concepts addressed and kinds of activities implemented. 
Workshops were well attended and appreciated for the knowledge, 
group support, and practical information that they provided and 
for the confidence that they instilled in participants. 

The review of project materials suggests that the district 
should continue to obtain literature that details how to 
incorporate diversities in student backgrounds, and life 
experiences in substance abuse prevention education. 

Finally, although a number of community-based organizations 
were involved in training activities, the district did not 
provide information on whether the proposed visits to substance 
abuse prevention sites took place. O.E.R. also received no 
information on the extent to which the district is developing a 
drug prevention curriculum, and each school is developing its own 
substance abuse prevention action plans. The proposed pre- and 
post- tests for students, staff and parents, and classroom 



observations for staff should be conducted in order to determine 
if there were in fact increases in knowledge gained, and if staff 
are able to translate staff development training into lessons for 
students. Finally, since staff did not specify position titles 
on workshop attendance sheets, O.E.R. cannot determine whether 
the pivotal objective of involving all aspects of the educational 
community in a preventive effort was met. 
RECOMMENDATIONS 

Based on the above findings, the following recommendations 
are made to meet project goals and enhance evaluation measures: 

• Efforts should be made to include more of the five new 
staff groups in staff training. 

• A structured checklist should be developed to evaluate 
project materials. 

• Project leaders should provide information on whether 
the staff actively mobilize parents in substance abuse 
prevention education. 

• The district should identify and develop a cluster of 
student characteristics to evaluate the impact of staff 
development on students. 

• Pre and post criterion referenced tests should be 
administered to evaluate the knowledge that staff 
obtained through program participation. 

• Staff development training attendance sheets should 
have a space designated for participants' job titles. 

• Staff trainees should be observed in order to assess 
the success of training in applied situations. 

• District staff should provide technical assistance to 
school staff so that the information from workshops 
can be readily applied in concrete situations. 

• Workshops should expand their emphasis on communication 
skills to enable staff members to relate more 
effectively with students and parents. 
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• Workshop time should be more effectively managed to 
better meet participants' needs. 

• The district should maintain documentation to determine 
if and how the New York State Education Dept. , the 
Northeast Regional Training Center , and local school 
selected community organizations were involved. 

• The district should provide information on how many of 
the schools are developing action plans for substance 
abuse prevention. 

• The district should provide a rationale for activities 
that differ with the activities outlined in the 
proposal. 




0 * 
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APPENDIX 



Program Outcomes and Participant Responses to Workshops 



Appendix Table 1 
Program Objectives and Outcomes 



Goals Ob j actives Outcomes* 



No. of staff trained 275 1120 b 

% able to demonstrate inc. 80 % c 

knowledge and improvement in 
prevention education 

No. of training events 

implemented 100 88 

No. of person training- 1,500 0 

days delivered 

No. of experts involved 60 54 

No. of community based 10 3 0 

organizations that assisted 
in training 

No. of student recipients 3,750 4965 d 

Each of the schools will 3 c 

develop action plans for SAP 



These numbers do not eliminate the overlapping that occurred 
when the same individual participated in more than one 
activity. 



b 



This number includes staff participating in training 
activities outlined in Table 2. 

Information not available. 

This figure includes instruction delivered by district SAPIS 
to 3,695 students, workshop participation by 143 students, 
1063 recipients of the extended-day program, and students 
mentored by Junior High School students. All attendance forms 
did not provide information on the grade levels of these 
students • 
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Appendix Table 2 
Number and Kind of Staff Targeted for Staff Development 



Staff Group 



Expected Number 
of Participants 



Actual Number 
of Participants 



Total school staff 

Principals and Ass. Principals 

Teachers 

Professional Support Staff 0 
Librarians and Teacher Trainers 
SAPIS 
Paras 

School secretaries & aides 
Custodial Workers 
Lunchroom Workers 
Schoolguards 



275 
16 

100 
24 
12 
4 
40 
19 
20 
20 
20 



1, 120 



137 b 



12 



* 



a 



k Data unknown. A breakdown according to staff titles was not 
provided on most attendance sheets. 

b There were 137 persons who attended staff development 
activities for guidance counsellors , social workers , 
psychologists , and nurses. These attendance forms did not 
allow for position title specification. 

0 This category includes guidance counsellors, social workers, 
psychologists, and nurses. 
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Appendix Table 4 



Teacher Assessment of the Orientation for the Summer Program 1 

Close-Ended Responses 



Workshop Qualities 


Response Choices 




Yes 


No 


Somewhat 


No 
Answer 


Workshop fulfilled 
expectations 


17 


3 


1 


3 


Leader provided 
useful information 


23 


0 


1 


0 


Activities were 
worthwhile 


20 


0 


4 


0 


Length of session 
was adequate 


20 


2 


2 


0 


Would recommend the 
workshop 


13 


4 


1 


1 


Will attend other 
workshops with the 
same format 


17 


5 


0 


2 



* The workshop was attended by twenty-four teachers. 
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Appendix Table 4 A 



Teacher Assessment of the Orientation for the Summer Program* 

Open Ended Responses 6 



Workshon Oualitips 


Responses 


Quality 


No. 


Most helnfu^ naTt of 
the workshop 


Stress Management 

relaxation exercises/ 
meditation 


12 




Group Activity 

sharincf knowledcre and Qunoort 
group; getting to know each 
other 


7 




Knowledge obtained 


5 




No response / Other 


4 


Recommended changes 


Chanoe in workshoo organization 
more structure and guided 
mediation; soeaker should have 
more command 


7 




No change 


9 




Other /Not applicable 


8 


Suggested future 
workshop topics 


Stress Management 
Conflict Resolution 


1 
1 




Parent Involvement 


1 




Working with children 


1 




Self-esteem building 


1 




Othei/ Not applicable 


18 



The workshop was attended by twenty-four teachers. 
Participants could give more than one response to a question. 
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